A deep neural network is a neural network with lots of layers.. Tyoically the layers are also large, in the sense of lots of nodes. Each layer can also have a differnt size and use different types of learning. For example, it is common for the first layer to be a Restricted Boltzmann Machine in order to perofrma dimensional reduction. TYpically the inner laters are underdetermined (many equally good arrangements of weightings); this and theor disrance from the output layer means that backpropogation or similar learning rules need to have very slow learning rates to avoid instabilities. In addition, more layers and laregr layers usually req=uires more tarng data. Together these mean that deep learning (training of deep neural networks) requires. a lot of training data. This combination of computational cost and data volume is one of the main reasons that the use deep neural networks were not widely adopted for many years.
Used on pages 10, 114, 122, 234, 274, 275, 518, 565
Deep learning architecture -- multiple layers, with varying connection topologies